<html>
<head><meta charset="utf-8"><title>benchmark for rust-analyzer#1331 · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html">benchmark for rust-analyzer#1331</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="166903455"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903455" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903455">(May 30 2019 at 12:35)</a>:</h4>
<p>So <span class="user-mention" data-user-id="133169">@matklad</span> if I wanted to do some benchmarking here, what would you suggest? I saw the regression test you pointed at for adding a newline, should I just try to convert that to a benchmark?</p>



<a name="166903467"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903467" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903467">(May 30 2019 at 12:35)</a>:</h4>
<p>yeah, that would be a good start!</p>



<a name="166903535"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903535" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903535">(May 30 2019 at 12:36)</a>:</h4>
<p>I am not sure we are really ready to adding "true" benchmarks to rust-analyzer's CI, but we certantly can write benches for one-off profiling</p>



<a name="166903548"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903548" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903548">(May 30 2019 at 12:37)</a>:</h4>
<p>Let me breifly explain how <a href="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L349-L411" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L349-L411">https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L349-L411</a> works.</p>



<a name="166903580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903580">(May 30 2019 at 12:37)</a>:</h4>
<p>it is an <strong>integration</strong> test, which creates a real instance of rust-analyzer, which reads real files from the hard drive</p>



<a name="166903658"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903658" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903658">(May 30 2019 at 12:38)</a>:</h4>
<p>In particular, it reads sources of standard library from the sysroot, so it can be used for load testing</p>



<a name="166903675"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903675" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903675">(May 30 2019 at 12:38)</a>:</h4>
<p><code>server</code> is basically a channel to which you can send LSP requests and read responses</p>



<a name="166903827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903827">(May 30 2019 at 12:41)</a>:</h4>
<p>for benchmarking salsa, I think what we should do is:</p>
<ul>
<li>open the proejct</li>
<li>wait until it is loaded</li>
<li>send <code>DidOpenTextDocument</code> request. This should trigger syntax highlighting and diagnosits</li>
<li>recieve diagnostics notification (it will take a while to compute one, especially in debug mode)</li>
<li>here's the point where we start profiling</li>
<li>send <code>DidChangeTextDocument</code> notification, which changes whitespace in the document</li>
<li>wait for the updated set of diagnostics.</li>
</ul>



<a name="166903956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166903956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166903956">(May 30 2019 at 12:42)</a>:</h4>
<p>The subtle bit here is that diagnostics computation is a side-effect of a change, it's not a direct response to some request. The logic which kicks diagnostic computation is here: <a href="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/src/main_loop.rs#L293-L301" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/src/main_loop.rs#L293-L301">https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/src/main_loop.rs#L293-L301</a></p>



<a name="166904007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166904007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166904007">(May 30 2019 at 12:43)</a>:</h4>
<p>You also would probably want to look at the logs/profiling info. The relevan env-vars look like this:</p>
<div class="codehilite"><pre><span></span>env RA_PROFILE=&#39;*&gt;16&#39; RUST_LOG=ra_lsp_server=info
</pre></div>


<p>(more infor about cryptic <code>*&gt;16</code> syntax in the <code>docs/dev/README.md</code>)</p>



<a name="166904062"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166904062" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166904062">(May 30 2019 at 12:44)</a>:</h4>
<p>You can set the logs in the tests:</p>
<p><a href="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L20-L22" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L20-L22">https://github.com/rust-analyzer/rust-analyzer/blob/6b88735fe6cd3b259816c7c90a2675ee057c9e4c/crates/ra_lsp_server/tests/heavy_tests/main.rs#L20-L22</a></p>



<a name="166905213"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166905213" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166905213">(May 30 2019 at 13:02)</a>:</h4>
<p>Oh, and a good way to benchmark from-scratch analysis is this command <code>cargo run --release -p ra_cli -- analysis-stats ../chalk/</code>. But yeah, for salsa we specifically interested in <em>incremental</em> analysis, and that's harder to set up</p>



<a name="166905254"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166905254" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166905254">(May 30 2019 at 13:03)</a>:</h4>
<p>that makes we wonder if we should add a <code>--diff</code> argument to the <code>analysis-stats</code> command, which allows one to check re-analysis as well?</p>



<a name="166980527"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980527" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980527">(May 31 2019 at 09:06)</a>:</h4>
<blockquote>
<p>that makes we wonder if we should add a <code>--diff</code> argument to the <code>analysis-stats</code> command, which allows one to check re-analysis as well?</p>
</blockquote>
<p>seems like a good idea, <span class="user-mention" data-user-id="133169">@matklad</span></p>



<a name="166980537"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980537" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980537">(May 31 2019 at 09:07)</a>:</h4>
<p>I didn't get a chance to do much yesterday but i'm tinkering with this a bit now</p>



<a name="166980553"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980553" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980553">(May 31 2019 at 09:07)</a>:</h4>
<p>I could imagine trying to add a <code>--diff</code> to that command -- but i'm first trying it out to see what it does =)</p>



<a name="166980558"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980558" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980558">(May 31 2019 at 09:07)</a>:</h4>
<p>Have you seen the benchmark setup we have for <code>perf.rust-lang.org</code>?</p>



<a name="166980563"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980563" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980563">(May 31 2019 at 09:07)</a>:</h4>
<p>(In particular, the patches)</p>



<a name="166980567"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166980567" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166980567">(May 31 2019 at 09:07)</a>:</h4>
<p>Might be nice to be able to be compatible with that at <em>some point</em></p>



<a name="166985552"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166985552" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166985552">(May 31 2019 at 10:31)</a>:</h4>
<p>No, I haven't seen that. I am somewhat skeptical that we can come up with really good benchmarks for latency of IDE features. </p>
<p>testing rustc with patch is easy: you need to check that full compilation is fast.</p>
<p>testing ra would be tricky, because you also have to select the subset of operations you want to do. The common scenario is something like "user types something, it kicks diagnostic processing, user asks for completion, and diagnostics and completion now race against each other". Setting up such scenarios is not just "invoke all of the queries".</p>
<p>What we can and should cerntaly do is to check that diagnostics and highlights for a single file after update have reasonable performance. (as diagostics and highlihgts exercise most of the queries). However, while this is the easiest thing to test, it also is the least useful one, as diagnostics can be computed in the background totally fine. </p>
<p>What we really interested in are real-time features, like completion or that dreaded on-enter handler. For those, I think investing in observability of the live server is more important than synthetic benchmarks (by the way, I <strong>love</strong> how <code>RA_PROFILE</code> shows real-time profiles, that's super useful)</p>



<a name="166987063"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166987063" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166987063">(May 31 2019 at 10:57)</a>:</h4>
<p>Also, it should probably be called <code>--patch</code> and not <code>--diff</code></p>



<a name="166992736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166992736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166992736">(May 31 2019 at 12:34)</a>:</h4>
<p>well I spent the morning just kind of reading into RA code and seeing how the pieces fit together</p>



<a name="166992744"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166992744" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166992744">(May 31 2019 at 12:35)</a>:</h4>
<p>I didn't actually do any measuring yet</p>



<a name="166992761"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166992761" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166992761">(May 31 2019 at 12:35)</a>:</h4>
<p>I certainly take your point that capturing the "real world" experience is complex</p>



<a name="166992794"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166992794" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166992794">(May 31 2019 at 12:35)</a>:</h4>
<p>Though it still seems to me that you could measure a lot of individual things, the total of which are important -- e.g., how fast can we do completions with base X and edit Y</p>



<a name="166992802"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166992802" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166992802">(May 31 2019 at 12:35)</a>:</h4>
<p>(and perhaps a stream of edits)</p>



<a name="166993142"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993142" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993142">(May 31 2019 at 12:41)</a>:</h4>
<p>It should be possible to do a 'typing' benchmark where we e.g. add a single letter and then request diagnostics, code actions and a 'real-time' request like onEnter or completions, right? and e.g. simulate typing a whole function that way</p>



<a name="166993228"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993228" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993228">(May 31 2019 at 12:42)</a>:</h4>
<p>Yeah, that should be possible.</p>



<a name="166993247"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993247" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993247">(May 31 2019 at 12:42)</a>:</h4>
<p>I also wonder if we could capture a trace of real interraction, and have a benchmark for "how fast can we replay the trace"</p>



<a name="166993342"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993342" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993342">(May 31 2019 at 12:44)</a>:</h4>
<p>Yeah, perhaps I should stop complaining that it's hard and .. just add a benchmark? Seems totally doable</p>



<a name="166993437"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993437" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993437">(May 31 2019 at 12:45)</a>:</h4>
<p>I guess, my experience with IntelliJ is harming me in this case: I haven't sloved benchmarking of IntelliJ Rust at all, but a significant part of that was that</p>
<ul>
<li>it's hard to do macro-benchmarks on JVM (we run benchmark like 20 times, picked the minimum time, and still got the wild variance)</li>
<li>IntelliJ platform has all sorts of caches you don't even know about, so it's hard to make sure you are benching the thing you <em>think</em> you are benching :)</li>
</ul>



<a name="166993496"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993496" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993496">(May 31 2019 at 12:46)</a>:</h4>
<p>Btw, <span class="user-mention" data-user-id="129457">@Florian Diebold</span> I must say that <code>analysis-stats</code> is an awesome tool!</p>



<a name="166993498"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/benchmark%20for%20rust-analyzer%231331/near/166993498" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/benchmark.20for.20rust-analyzer.231331.html#166993498">(May 31 2019 at 12:46)</a>:</h4>
<p>Thanks!</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>